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Figure 10: Schematic representation of method for creating an extended 
virtual sound source from a plurality of virtual point sources derived from 
a single original source using decorrelation. 
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1 

A METHOD OF SYNTHESISING AN AUDIO SIGNAL 

This invention relates to a method of synthesising an audio signal having 
left and right channels corresponding to a virtual sound source at a given 
5 apparent location in space relative to a preferred position of a listener in use, the 
information in the channels including cues for perception of the direction of said 
virtual sound source from said preferred position. 

The processing of audio signals to reproduce a three dimensional sound- 
field on replay to a listener having two ears has been a goal for inventors since the 

10 invention of stereo by Alan Blumlein in the 1930's. One approach has been to use 
many sound reproduction channels to surround the listener with a multiplicity of 
sound sources such as loudspeakers. Another approach has been to use a dummy 
head having microphones positioned in the auditory canals of artificial ears to 
make sound recordings for headphone listening. An especially promising 

15 approach to the binaural synthesis of such a sound-field has been described in EP- 
B-0689756, which describes the synthesis of a sound-field using a pair of 
loudspeakers and only two signal channels, the sound-field nevertheless having 
directional information allowing a listener to perceive sound sources appearing to 
lie anywhere on a sphere surrounding the head of a listener placed at the centre of 

20 the sphere. 

A drawback with such systems developed in the past has been that 
although the recreated sound-field has directional information, it has been 
difficult to recreate the perception of having a sound source which is perceived to 
move towards or away from a listener with time, or that of a physically large 

25 sound source. 

According to a first aspect of the invention there is provided a method as 
specified in claims 1-11. According to a second aspect of the invention there is 
provided apparatus as specified in claim 12. According to a third aspect of the * 
invention there is provided an audio signal as specified in claim 13. 

30 It might be argued that to synthesise a large area sound source one might 

use a large area source for a particular HRTF measurement. However, if a large 
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loud spea k er is used for the HRTF measurements, .hen the resu* are gross and 
loudspe .vrnxp amplitude characteristics become meaningless. 

element of the HRTF (Figure 1), which is a critical parameter. The results are 

^^entso^einvenuonwiUnowhedescriW.hywayofexamp, • 
only , with reference to the accompanying diagrammatic drawings, in which 
Figure 1 shows a prior art method of synthesising an audio signal, 

Figure 2 shows a real extended sound source, 
Fieure 3 shows a second real extended sound source, 

H^showsabloc^diagramofmemodsof synthesis for a) headphone and b) 

' P^rshowsabloclcdiagramofafirsten^odimentaccordingtothemvenhon, 

Figure 7 shows a comb filter and its characteristics, 
Figure 8 shows a pair o, complimentary comb filter 
FigL 9 shows a triplet sound source using complimentary comb filters, 
, 0 Figure 10 shows a second embodiment according to the invention, 
' Hgure U shows a third embodiment according to the invention, 
Figure 12 shows the recreation of the sound source of Figure 2. 
Figure 13 shows a fourth embodiment of the invention 

u .^Hiaeram of a known method of simulating a 
Figure 14 shows a schematic diagram «<" 

according to the present invention. 

TKe present invention relates particularly to the reproduction 
30 from two-speaW stereo systems or headphones. Tnis type of 3D-sound is 




3 

described, for example, in EP-B-0689756 which is incorporated herein by- 
reference. 

It is well known that a mono sound source can be digitally processed via a 
pair of "Head-Response Transfer Functions" (HRTFs), such that the resultant 

5 stereo-pair signal contains 3D-sound cues. These sound cues are introduced 
naturally by the head and ears when we listen to sounds in real life, and they 
include the inter-aural amplitude difference (IAD), inter-aural time difference 
(TTD) and spectral shaping by the outer ear. When this stereo signal pair is 
introduced efficiently into the appropriate ears of the listener, by headphones say, 

10 then he or she perceives the original sound to be at a position in space in 

accordance with the spatial location of the HRTF pair which was used for the 
signal-processing. 

When one listens through loudspeakers instead of headphones, then the 
signals are not conveyed efficiently into the ears, for there is "transaural acoustic 

15 crosstalk" present which inhibits the 3D-sound cues. This means that the left ear 
hears a little of what the right ear is hearing (after a small, additional time-delay 
of around 0.2 ms), and vice versa. In order to prevent this happening, it is known 
to create appropriate "crosstalk cancellation" signals from the opposite 
loudspeaker. These signals are equal in magnitude and inverted (opposite in 

20 phase) with respect to the crosstalk signals, and designed to cancel them out. 
There are more advanced schemes which anticipate the secondary (and higher 
order) effects of the cancellation signals themselves contributing to secondary 
crosstalk, and the correction thereof, and these methods are known in the prior 
art. 

25 When the HRTF processing and crosstalk cancellation are carried out 

correctly, and using high quality HRTF source data, then the effects can be quite 
remarkable. For example, it is possible to move the virtual image of a sound- 
source around the listener in a complete horizontal circle, beginning in front, 
moving around the right-hand side of the listener, behind the listener, and back 

30 around the left-hand side to the front again. It is also possible to make the sound 
source move in a vertical circle around the listener, and indeed make the sound 
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p from any selected position in space. However, some particular 

^ . . u a KWhfcause there is more left-nghtdinerence 
• frnnt ^azimuth = 0°). This is probably because u«u 

m fr ° nt (aZ . ... Similar l y/ it is difficult to differentiate 

inforrmHonforthebramtoworkv^m.Warly, 

between a sound source directly in front of the listener (azunuth - 0 ) and a 
between fV ,_iRcn This is because there is no 

source direclybawnd ^listener (az,muth- 18 0 ). ™" 

• ^ T««rarHce there is more HF energy perceive 

u • «hv««cal limit on the minimum size of loudspeaker 
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measured at 1 metre and those measured at much greater distances, the 1 metre 
measurement is used. 

The effect of a sound source appearing to be in the mid-distance (1 to 5 m, 
say) or far-distance (>5 m) can be created easily by the addition of a reverberation 
5 signal to the primary signal, thus simulating the effects of reflected sound waves 
from the floor and walls of the environment. A reduction of the high frequency 
(HF) components of the sound source can also help create the effect of a distant 
source, simulating the selective absorption of HF by air, although this is a more 
subtle effect. In summary, the effects of controlling the apparent distance of a 

10 sound source beyond several metres are known. 

Alternatively, in many PC games situations it is desirable to have a sound 
effect appear to be very close to the listener. For example, in an adventure game, it 
might be required for a "guide" to whisper instructions into one of the listener's 
ears, or alternatively, in a flight-simulator, it might be required to create the effect 

15 that the listener is a pilot, hearing air-traffic information via headphones. In a 
combat game, it might be required to make bullets appear to fly close by the 
listener's head. These effects are not possible solely using HRTFs measured at 1 
metre distance, but they can be synthesised from 1 metre HRTFs by additional 
signal-processing to re-create appropriate differential L-R sound intensity values, 

20 as is described in our co-pending patent application GB9726338.8 which is 
incorporated herein by reference. 

In all of the prior art, the virtual sound sources are created and represented 
by means of a single point source. At this stage, it is worth defining what is meant 
here, in the present document, by the expression "virtual sound source". A virtual 

25 sound source is a perceived source of sound synthesised by a binaural (two- 
channel) system (i.e. via two loudspeakers or by headphones), which is 
representative of a sound-emitting entity such as a voice, a helicopter or a 
waterfall, for example. The virtual sound source can be complemented and 
enhanced by the addition of secondary effects which are representative of a 

30 specified virtual environment, such as sound reflections, echoes and absorption, 
thus creating a virtual sound environment. 
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The present invention comprises a means of 3D-sound synthesis for 
mating virtua! sound images with improved reaUsm compared to *e pnor art. 

point sources, rather than from a single, point source as is presendy By 
distributing said plurality of virtual sound sources over a P"-*^" 

truly representative of the real physical srtuation. The plurahty of vutu 
rcauLdtomain^consUntrelauvep^andsowhenmeyare^deto 

approach or leave the listener, the apparent size of the virtua! sound-enutung 
~Ki^t rhanees iust as it would if it were real. 

from a plurality of dissimilar virtual point source, Again, this * «*~*~™ 
IJ-Hfe — andmeresu.tUU.enhance^reaUsmofasynmes^ 

virtual sound image. .„. W( , h 
Finally, i, is wo* noting that mere is a particu>ar, relevant effect whKh 
c^s when synthesising 3D sound which must be taken into account. Wh» 

Tre is a large common-mode content present between left and ngh, charts, 
other even number, of virtual sounds are to be synthesised in a symmetncal 

25 ^eUtionbe^eenme^viduaisoundsources^dmeresult.thatthe 

lived sounds can become "fused" together into one. A means of pre^ting 
Tlucing mis effect is to create two or more decollated sources from any 

sound image from two or more virtua. point sources of sound, 
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(a) where the plurality of point sources are similar, but the different HRTF 
processing applied to them decorrelates them sufficiently so as to be 
separately distinguishable without further decorrelation; 

(b) where a decorrelation method is used to create a plurality of sound sources 
from a single original sound source (this is especially useful where the 
sounds are to be placed symmetrically about the median plane); 

(c) where the plurality of sounds are derived from different sources, each 
representative of an element of the real-life sound source which is being 
simulated. 

The emission of sound is a complex phenomenon. For any given sound 
source, one can consider the acoustic energy as being emitted from a continuous, 
distributed array of elemental sources at differing locations, and having differing 
amplitudes and phase relationships to one another. If one is sufficiently far 
enough from such a complex emitter, then the elemental waveforms from the 
individual emitters sum together, effectively forming a single, composite wave 
which is perceived by the listener. It is worth defining several different types of 
distributed emitter, as follows. 

Firstly, a point source emitter. In reality, there is no such thing as a point 
source of acoustic radiation: all sound-emitting objects radiate acoustic energy 
from a finite surface area (or volume), and it will be obvious that there exists a 
wide range of emitting areas. For example, a small flying insect emits sound from 
its wing surfaces, which might be only several square millimetres in area. In 
practise, the insect could almost be considered as a point source, because, for all 
reasonable distances from a listener, it is clearly perceived as such. 

Secondly, a line source emitter. When considering a vibrating wire, such as 
a resonating guitar string, the sound energy is emitted from a (largely) two 
dimensional object: it is, effectively, a "line" emitter. The sound energy per unit 
length has a maximum value at the antinodes, and minimum value at the nodes. 
An observer close to a particular string antinode would measure different 
amplitude and phase values with respect to other listeners who might be equally 
close to the string, but at different positions along its length, near, say, to a node 



or the nearest adjacent antinode. At a distance, however, the elemental 

with spatial positionbecause of the differing path lengths to the elemental 
emitters (and hence differing phase relationships). 

Thirdly, an area source emitter. A resonating panel is a good example of an 
area source. As for the guitar string, however, the area wUl possess nodes . 
abodes according to its mode of vibration at any given fluency, and these 
summate at sufficient distance to form, effectively, a single wave^ 

Fourthly, a volume source emitter. In contrast to the msec, pom, source , 
0 a waterfall cascading on to roCcs might emit sound from a vo.^^ 
m ousands o, cubic metres in size: the waterfall is a very large : volume sou™. 
However, if it were a great distance from the listener (but still wrtlun heanng 

15 material in the bulk of the volume. 

to a practical situation, what are the important Ksues « deadmg whethe 
real distributed emitter can be considered to be a point source, or whether r, 
shlle synthesised as a more complex, distributed source, The factor wmch 
d^gl- tether a perceived sound source is similar to a point source or no, 
20 t tZg le subtended by the sound-emitting area at the head of the Ustener. In 
Ccal terms, this is related to our ability to perceive ma, an emitting ob.ec, to* 

as the insec,. I, has been shown by A W Mills 0- Acous,. Soc. Am. 1958 vol 30, 
Le 4 pages 237 - 246) ma, the "minimum audible angle" corresponds ,o an 

Icremen,al azimuth angle of abou, !,«(a, 0» azimum and elevation). . pmctica, 
.erms. we have found i, appropriate ,o use an increment azimum unr, of , 
be cause mis is suf fidenUy small as ,o be almost indiscerrtb.e when movmg a 
virtual sound source from one poin, to another, and a*o me as^ed time 
30 corresponds approximately to one sample period (a, 44, kHz fluency). 
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However, these values relate to differential positions of a single sound source, 
and not to the interval between two concurrent sources. 

From experiments, the inventor believes that a sensible method for 
differentiating between a point source and an area source would be the 
5 magnitude of the subtended angle at the listener's head, using a value of about 
20° as the criterion. Hence, if a sound source subtends an angle of less than 20° at 
the head of the listener, then it can be considered to be a point source; if it 
subtends an angle larger than 20°, then it is not a point source. 

As an extension of the principle of synthesising a virtual sound source from 
10 a plurality of sound sources where the sources derive from one original source, 
such as a .WAV computer file, an alternative approach exists where the sound 
sources may be different to each other. This is a powerful method of creating a 
virtual image of a large, complex sound-emitting object such as a helicopter, 
where a number of individual sources can be identified. For example, Figure 2 
15 shows a diagram of a helicopter showing several primary sound sources, namely 
the main blade tips, the exhaust, and the tail rotor. Similarly, Figure 3 shows a 
truck with the main sound-emitting surfaces similarly marked: the engine block, 
the tyres and the exhaust. In both cases it would be advantageous to create a 
composite sound image of the object by means of a plurality of individual virtual 
20 sound sources: one for the exhaust, one for the rotor, and so on. In a computer 

game application, the game itself links the individual sources geometrically, such 
that when they are relatively distant to the listener, they are effectively 
superimposed on each other, but when they are close up, they are physically 
separated according to the pre-arranged selected geometry and spatial positions. 
25 An important consequence of this is that a virtual sound source which is thus 

created scales with distance: it appears to increase in size when it approaches, and 
diminishes when it goes away from the listener. Also, when this sound source is 
caused to be "dose" to the listener, it appears convincingly so, unlike prior-art 
systems where a point source would be used to create a virtual image of all 
30 objects, irrespective of their physical size or the angle which they should subtend 
at the preferred position of the listener. 
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Figure 1 shows a block diagram of the HRTF-based signal-processing 
method which is used to create a virtual sound source from a mono sound source 
(such as a sound recording, or via a computer from a .WAV file or similar). The 
methods are well documented in the prior art, such as for example EP-B-0689756. 
i Figure 1 shows that left- and right-channel output signals are created, which, 
when transmitted to the left and right ears of a listener, create the effect that the 
sound source exists at a point in space according to the chosen HRTF 
characteristics, as specified by the required azimuth and elevation parameters. 
Figure 4 shows known methods for transmitting the signals to the left and 
0 right ears of a listener, first, by simply using a pair of headphones (via suitable 
drivers), and secondly, via loudspeakers, in conjunction with transaural crosstalk 
cancellation processing, as is fully described in WO 95/15069. 

Consider, now, for example, the situation where it is required to create the 
effect of a large truck passing the listener at differing distances, as depicted in 
L5 Figure 5. At a distance, a single point source is sufficient to simulate the truck. 
However, at close range, the engine enclosure panels emit sound energy from an 
area which subtends a significant area at the listener's head, as shown, and it is 
appropriate to use a plurality of virtual sources, as shown schematically in Figure 
6. (Figure 6 also shows the crosstalk cancellation processing appropriate for 
20 loudspeaker listening, as described above.) 

In many circumstances, especially when virtual sound effects are to be 
recreated to the sides of the listener, the HRTF processing decollates the 
individual signals sufficiently such that the listener is able to distinguish between 
them and hear them as individual sources, rather than "fuse" them into 
25 apparently a single sound. However, when there is symmetry in the placement of 
the individual sounds (say, one is to be placed at -30° azimuth in the horizontal 
plane, and another is to be placed at +30°), then our hearing processes cannot 
distinguish them separately, and create a vague, centralised image. 

This is consistent with reality, where the individual elemental sources 
30 which make up a large area sound source all possess differing amplitude and 
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phase characteristics, whereas in practise, we are often obliged to use a single 
sound recording or computer file to create the plurality of virtual sources for the 
sake of economy of storage and processing. Consequently, there is an 
unrealistically high correlation between the resultant array of virtual sources. 
5 Hence, in order to improve the effectiveness of the invention, there is preferably 
provided the ability to decorrelate the individual signals. In order to rrunimise the 
signal processing requirements (and irunimise costs and processing complexity), it 
is advantageous to use simple methods. The following method has been found to 
be an example of an effective, simple means of decorrelation, applicable to the 
10 present invention. 

A signal can be decorrelated sufficiently for the present invention by means 
of comb-filtering. This method of filtering is known in the prior art, but has not 
been applied to 3D-sound synthesis methods to the best of the applicants 
knowledge. Figure 7 shows a simple comb filter, in which the source signal, S, is 
15 passed through a time-delay element, and an attenuator element, and then 
combined with the original signal, S. At frequencies where the time-delay 
corresponds to one half a wavelength, the two combining waves are exactly 180° 
out of phase, and cancel each other, whereas when the time delay corresponds to 
one whole wavelength, the waves combine constructively. If the amplitudes of the 
20 two waves are the same, then total nulling and doubling, respectively, of the 
resultant wave occurs. By attenuating one of the combining signals, as shown, 
then the magnitude of the effect can be controlled. For example, if the time delay 
is chosen to be 1 ms, then the first cancellation point exists at 500 Hz. The first 
constructive addition frequency points are at 0 Hz, and 1 kHz, where the signals 
25 are in phase. If the attenuation factor is set to 0.5, then the destructive and 

constructive interference effects are restricted to -3 dB and + 3 dB respectively. 
These characteristics are shown in Figure 7 (lower), and have been found useful 
for the present purpose It might often be required to create a pair of 
decorrelated signals. For example, when a large sound source is to be simulated in 
30 front of the listener, extending laterally to the left and right, a pair of sources 

would be required for symmetrical placement (e.g. -40° and +40°), but with both 
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screes individually distinguishable. This can be done efficient* by creating and 
using a pair of complementary con* filters. This is achieved, firstly, by creating 
an identical pair of filters, each as shown according to Figure 7 (and with identical 
time delay values), but with signal inversion in one of the attenuation pathway, 
Inversion can be achieved either by (a) changing the sumrrdng node to a 
"differencing" node (for signal subtraction), or (b) inverting the attenuate 
coefficient (e.g. from + 0.5 to -0.5); the end result is the same in both case. The 
outputs of such a pair of complementary filters exWbit maximal ampbtude ■ 
decollation within the constraints of the attenuation factors, because the peaks 
ofonecorrespondto*e tt oughsofu,eo«her(Figure8),andviceversa. 

If a source "triplet" were required, then a convenient method of creating 
such an arrangement is shown in Figure 9, where a pair of maximally 
decollated sources are created, and then used in conjunction with the ongmal 
source itself, thus providing three decollated sources. 

Accordingly, a general system for creating a plurality of n P°™* 
from a sound source is shown in Figure 10. In such a situation, it canbe rnefhoen. 
to reproduce the low-frequency (LF) sound components from all of the elemental 
sound sources because (a, LF sounds can not be "localised" by human hearmg 
systems, and (b) LF sounds from a real source will be largely in V^*^ e ^ tion^t 

might be advantageous to supply the LF via the primary channel, and apply LF 
cut filters to the decollation channels (Figure 11). 

As mentioned previously, many real-world sound sources can be broken 
down into an array of individual, differing sounds. For example, a helicopter 
5 generates sound from several sources (as shown previously in Figure 2), mcludmg 
L blade tips, the exhaust, and tine tail-roto, If one were to create a vrrtual sound 
source representing a helicopter using only a point source, it would appear hke 
recording of a helicopter being replayed through a small, invisible loudspeaker 
rath er than a real helicopter. If, however, one uses tine present invent™ ,o create 
30 such an effect, it is possible to assign various different virtual sounds for each 
source (blade tips, exhaust, and so on), linJced geometrical* in virtual space to 
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create a composite virtual source (Figure 12), such that the effect is much more 
vivid and realistic. The method is shown schematically in Figure 13. There is a 
significant added benefit in doing this, because when the virtual object draws 
near, or recedes, the array of virtual sound sources similarly appear to expand 

5 and contract accordingly, which further adds to the realism of the experience. In 
the distance, of course, the sound sources can be merged into one, or replaced by 
a single point source. 

The present invention may be used to simulate the presence of an array of 
rear speakers or "diffuse" speaker for sound effects in surround sound 

10 reproduction systems, such as for example, TUX or Dolby Digital (AC3) 

reproduction. Figures 14 and 15 show schematic representations of the synthesis 
of virtual sound sources to simulate real multichannel sources, Figure 14 showing 
virtual point sound sources and Figure 15 showing the use of a triplet of 
decorrelated point sound sources to provide an extended area sound source as 

15 described above. 

Although in the above embodiments all the Figures show the presence of 
transaural crosstalk cancellation signal processing, this can be omitted if 
reproduction over headphones is required. 

Finally, the content of the accompanying abstract is hereby incorporated 

20 into this description by reference. 
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CLAIMS 



A method of synthesising an audio signal having left and right channels 
corresponding to a virtual sound source at a given apparent location in 
space relative to a preferred position of a listener in use, the information in 
the channels including cues for perception of the direction or relative 
position of said virtual sound source from said preferred position, 
characterised in that the virtual sound source is an extended source which 
comprises a plurality of point sources, the sound from each point source 
being spatially related to the sound from the other point sources 
comprising the extended virtual sound source, such that sound appears to 
be emitted from a region of space having a non-zero extent in one or more 
dimensions, the method including the steps of :- 

a) choosing one or more single channel signals for synthesising a plurality 
15 of point sound sources comprising the virtual sound source; 

b) defining the required spatial relationships between the plurality of point 
sound sources relative to one another; 

c) selecting the apparent locations for the point sound sources comprising 
the virtual sound source relative to said preferred position at a given time; 

d) processing the signal corresponding to each point sound source to 
provide left and right channel signals for each point sound source, the 
processed signals including cues for perception of the apparent direction or 
relative position of said point sound source from said preferred position; 

e) combining the plurality of left channel signals and combining the 
plurality of right channel signals to provide an audio signal having left and 
right channels corresponding to the said virtual sound source. 
A method of synthesising an audio signal as claimed in claim 1 in which 
the plurality of point sound sources include two or more sources having 
substantially identical signals, the signals being modified to be sufficiently 
different from one another to be separately distinguishable by a listener 
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when the two or more sources are disposed symmetrically on either side of 
the said preferred position. 

A method as claimed in claim 2 in which the modification is performed 
before step d). 

A method as claimed in claim 2 or 3 in which the modification of said two 
or more substantially identical signals comprises or includes filtering one 
or more of said signals using one or more respective decorrelation filters. 
A method as claimed in claim 4 in which the one or more respective 
decorrelation filters comprise comb filters. 

A method as claimed in any preceding claim in which the plurality of point 
sound sources represent sounds travelling directly from the apparent 
position of the virtual sound source to the said preferred position which 
are not reflected sounds or reverberant sound. 

A method as claimed in any preceding claim in which step d) comprises 
providing a left channel and a right channel having the same signal in both, 
modifying each of the channels using a respective head related transfer 
function to provide a signal for the left ear of a listener in the left channel 
and a signal for the right ear of a listener in the right channel, and 
introducing a time delay between the channels corresponding to the inter- 
aural time difference for a signal coming from the selected apparent 
direction or position of the corresponding point sound source relative to 
said preferred position. 

A method as claimed in any preceding claim in which the left signal and 
the right signal are compensated to cancel or reduce transaural crosstalk 
when supplied as left or right channels for replay by loudspeakers remote 
from the listener's ears. 

A method as claimed in any preceding claim in which the resulting two 
channel audio signal is combined with a further two or more channel 
signal. 
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A method as claimed in claim 9 in which the signals are combined by 
adding the content of corresponding channels to provide a combined 
signal having two channels. 

A method as claimed in any preceding claim in which the apparent 
locations for the point sound sources comprising the virtual sound source 
relative to said preferred position are selected such as to change with tune 
to give the impression of movement of the virtual sound source. 
Apparatus for performing a method as claimed in any preceding daun. ■ 
An audio signal processed by a method as claimed in any preceding daun. 



